1. Field of the Invention
The present invention relates generally to the processing of video images and, more particularly, to techniques for detecting and smoothing diagonal features in video images.
2. Description of the Related Art
All major television standards use a raster scanning technique known as xe2x80x9cinterlacingxe2x80x9d or xe2x80x9cinterlace scanning.xe2x80x9d Interlace scanning draws horizontal scan lines from the top of the screen to the bottom of the screen in two passes. Each pass is known as a field. In the National Television System Committee (NTSC) standard used in North America, each field takes approximately {fraction (1/60)}th of a second to draw.
Interlace scanning depends of the ability of the cathode ray tube (CRT) phosphors to retain an image for a few milliseconds, in effect acting like a xe2x80x9cmemoryxe2x80x9d to retain the previous field while the newer interleaved field is being scanned. Interlace scanning provides a benefit in television systems by doubling the vertical resolution of the system without increasing broadcast bandwidth.
FIG. 1 shows a number of parallel horizontal scan lines 10 on a conventional television display. A first set of horizontal lines 12 is scanned in a first field period and then a second set of horizontal lines 14 is scanned in a second field period. Thus, the first field is temporarily shifted by {fraction (1/60)}th of a second from the second field. When rapidly changing images are being displayed, an object in motion may appear to be fuzzy due to the temporal displacement between the two fields.
This temporal displacement typically does not create a problem on conventional television displays, primarily because the image of the xe2x80x9colderxe2x80x9d field quickly fades in intensity as the light output of the phosphors decays. A secondary reason is that the spatial displacement in the images caused by motion results in a fine detail that television displays resolve well. For these reasons, interlace scanning of motion pictures works acceptably well on conventional television displays.
FIG. 2 shows a set of progressively scanned horizontal lines 16. In progressive scanning, all horizontal lines 16, are scanned out in one vertical pass 18, so there is no time displacement of adjacent lines as in interlace scan. Progressive scanning requires a much higher bandwidth signal. Consequently, progressive scanning is typically used for applications where improved image quality and higher resolution are required, relative to conventional television systems. Progressive scanning is widely used in computer CRTs and liquid crystal displays (LCD).
If a motion picture formatted for an interlaced monitor device as in FIG. 1 is to be displayed on a progressively scanned device as in FIG. 2, then it must be converted from the interlaced format to the progressive format. This format conversion is known as deinterlacing. FIG. 3 is a flow diagram of a deinterlace process 19 of the prior art. A first series of interlaced video fields 20 is generated by a video source (not illustrated) at {fraction (1/60)}th second intervals.
In this example, each of the video fields 20 has a spatial resolution of 720 horizontal by 240 vertical pixels. Each field contains half the vertical resolution of a complete video image. The first series of video fields 20 are input to a deinterlace processor 22, which converts the 720 by 240 interlaced format to a second series of video fields 24. In this example, each of the second series of video fields 24 may have 720 by 480 pixels where the fields are displayed at 60 frames per second.
FIG. 4 shows a prior art method 25 of deinterlace processing. A video field 26 containing scan lines 30, and a previous video field 28 containing scan lines 32 is fed into a field combination deinterlace processor 34. The result is a combined frame 36 with scan lines 38 sourced from video field 26 and scan lines 40 sourced from video field 28. When this simple deinterlacing of the prior art is performed, and a motion picture formatted for an interlace display is converted to a progressive format, a noticeable xe2x80x9cartifactxe2x80x9d or error arises because the image content of vertically adjacent lines is time shifted by {fraction (1/60)}th second as noted previously. The error is most visible around the edges of objects that are in motion.
FIG. 5 shows a deinterlaced image 42 with a stationary object 43 that is rendered without distortion. FIG. 6 shows an image 44 with the object 43xe2x80x2 in motion. The edges of object 43xe2x80x2 create artifacts 45 on the edges of the image 44 because of the aforementioned temporal shift. These artifacts 45 are introduced into the image by the conventional field combination deinterlacing method 25 of FIG. 4.
FIG. 7 is an illustration of an alternative prior art method 46 to deinterlace an image using a single reference field rather than two fields. The method 46 interpolates or doubles the number of lines of one field to produce a progressive frame. A video field 48 is scanned from an image to contain a half set of lines 50. The half set of lines 50 is deinterlaced by line interpolation in a deinterlacing interpolator 52.
The resulting frame 54 will have all the lines 50 of the original video field 48. The remaining lines 56 are created by interpolation of lines 50. The resultant image will not have motion artifacts because all the lines in the image will be created from lines 50 that are time correlated. This alternative method 46 of deinterlacing does not produce motion artifacts, but the vertical resolution of the image is reduced by half.
Reduction in vertical resolution is particularly noticeable in areas within the image that have high contrast diagonal features. In this case, the reduction in vertical resolution results in a jagged appearance to diagonal image features. FIG. 8 illustrates a conventional two-dimensional array of pixels 58 in which a high contrast diagonal feature exists. This array 58 is the output of a deinterlace processor. The lines numbered 0, 2, 4, 6, and 8 come from one original video field, and lines 1, 3, 5, and 7 come from the previous original video field.
If a motion artifact is detected in the region of these pixels, then the deinterlace processor will discard the pixels from the previous field in lines 1, 3, 5, and 7. The array 60 containing the remaining pixels in lines 0, 2, 4, 6, and 8 are shown in FIG. 9. The deinterlace processor will then compute the missing pixels from the lines shown in FIG. 9 producing a very jagged image 62 as shown in FIG. 10.
In summary, prior art deinterlacing methods that operate based upon interpolation reduce the vertical resolution of the original image. This reduction in resolution is particularly noticeable in images with high contrast diagonal features. In view of the foregoing, it is desirable to have a method that detects diagonal features and smoothens the jagged appearance caused by a reduction in resolution along diagonal features in areas where deinterlace processing takes place.
The present invention fills these needs by providing an efficient and economical method and apparatus for detecting and smoothing high contrast diagonal features in video images. It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a device or a method. Several inventive embodiments of the present invention are described below.
In one embodiment of the present invention, a digital image processor is provided. The digital image processor includes a deinterlacing processor coupled between an input buffer operable to receive an interlaced video stream and an output operable to transmit a deinterlaced video stream. The deinterlacing processor is also coupled to a digital memory for storing portions of the interlaced video signal. The deinterlacing processor is operable to detect said diagonal features in the portions of the received interlaced video stream and to generate the deinterlaced video stream having smoothed diagonal features.
In another embodiment of the present invention, a method for deinterlacing an interlaced video stream is provided. The method includes receiving a video frame including a number of pixels from an input of the interlaced video stream. The video frame is analyzed for frequency information inherent to the video frame in order to detect motion artifacts and the magnitude of the motion artifacts in the pixels in the video frame. Diagonal features surrounding the pixels in the video frame are detected if a motion artifact is detected. Each pixel is then mixed with a set of spatially corresponding pixels to generate an output pixel, while using the magnitude of the motion artifacts as a control, to generate an output pixel.
In another embodiment of the present invention, a method for deinterlacing an interlaced video stream is provided. The method includes receiving a video frame including a number of pixels from an input of the interlaced video stream. The video frame is analyzed for frequency information inherent to the video frame in order to detect motion artifacts. A number of motion artifact detection values is determined for the pixels in the video frame. A magnitude for the plurality of motion artifact detection values is then determined. Diagonal features surrounding the pixels in the video frame are detected if a motion artifact is detected. Each pixel is then mixed with a set of spatially corresponding pixels to generate an output pixel, while using the magnitude of the motion artifacts as a control, to generate an output pixel.
An advantage of the present invention is that it allows for detection and smoothing of high contrast diagonal features that result from deinterlacing video images. By reducing the effect of the diagonal features, the processed video image becomes clearer and much less jagged.
Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.